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Abstract 

Various Web 2.0 technologies can be used to support pedagogy. Examples include wikis, blogs, and 
social media including forum discussions. Online class forum discussions involving electronic text can 
result in robust strings of data containing meta-knowledge, inherent meaning, themes and patterns. 
Based on instructional design, learning outcomes guide and reflect class generated work product such 
as assignments, activities, and discussions. As such, class discussions should evolve with alignment to 
learning outcomes. One measurement of instructional efficacy involves the closeness with which this 
alignment occurs. In this experimental research the authors report on the design and prototyping of a 
deterministic model utilizing a tag cloud engine to determine dominant and emerging themes from a 
text string, namely word data collected from a threaded discussion. Textual data used in this 
investigation involved two Information Systems online classes where threaded discussions during one 
week were captured as a text string. Text from a learning management system threaded discussion 
was fed into a tag cloud engine where emerging and dominant asynchronous conversation themes 
were determined. Calculating a correlation coefficient as an indicator of pedagogical efficacy, the 
application evaluated the pedagogical efficacy evidenced in the discussion forum through comparison 
of themes with instructional objectives. In this experimental research, a real-time online analytical 
processing (OLAP) tool prototype to support pedagogical intelligence via systemic formative evaluation 
was designed and developed. Findings from the investigation were used to reach conclusions 
regarding the use of Web 2.0 technologies in guiding instruction. 

Keywords: Web 2.0, eLearning, tag cloud, word cloud, correlation, coefficient 
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1. INTRODUCTION 

This study represents a continuation of an 
existing line of inquiry (Conn, Hu, Boyer, and 
Wilkinson, 2009) involving the use of Web 2.0 
technologies in curriculum and instruction. 
Motivation for this study also is supported by 
consistent advances in eLearning tools, 
technologies, and applications. Higher education 
institutions continue to engage in eLearning 
initiatives at an accelerating rate. According to 
Sloan Consortium [Sloan-C] (2009), by the end 
of the decade the number of students taking 
courses online is expected to grow to over 2.6 
million. Moreover, 40.7% of institutions offering 
eLearning courses found that students have an 
equivalent level of satisfaction with online 
instruction, 56.2% had no opinion, and a minor 
population (3.1%) was not satisfied with online 
instruction. 

eLearning courses, as a percent of the overall 
market for education and training, now exceed 
10% of the total. The trend toward fully online 
programs continues to increase; two-thirds of 
the largest United States (U.S.) higher education 
institutions now have fully online programs 
(Sloan-C, 2009). Moreover, a traditional 
learning management system (LMS) is no longer 
able to keep pace with advanced Internet 
technologies and increasing e-Learner 
requirements (Dagger, O'Connor, Lawless, 
Walsh, & Wade, 2007). Of concern is seamless 
information interoperability in the eLearning 
platform; thus, the monolithic architecture of the 
LMS is not an accommodating solution (Dagger 
et al.). Although this research was conducted 
primarily with an Information Systems online 
eLearning student environment in mind, the 
research also is applicable to ground-based 
classroom environments and other disciplines 
where technology is available to facilitate and 
mediate instruction. 

The term tag cloud refers to a visualization of 
word data based on a scheme of relevance, 
importance, or popularity represented by 
manipulated visual properties such as font size, 
color, intensity, width, position, or weight 
(Bateman, Gutwin, & Nacenta, 2008). According 
to Xexeo, Morgado, and Fiuza (2009), many new 
approaches to the use of tag clouds exist. In 
this experimental research specific Web 2.0 
technology, namely a tag cloud engine, was 
used to generate outputs that were evaluated 
using Spearman's rho (p), a rank correlation 
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coefficient. The researchers sought to construct 
a deterministic model using prototyping. 
Initially, conceptually applying various methods 
to examine relationships, the researchers 
considered use of: 


(i) 

functions to determine if a 
relationship exists. 

causal 

(ii) 

regression analysis. 


(iii) 

pattern assessment 

scatterplots. 

using 

(iv) 

reasoning under uncertainty 
probability, and 

using 

(V) 

correlation coefficient. 



After consideration of the phenomena between 
bivariates, the researchers selected a coefficient 
of correlation as the estimating equation. This 
selection provided confirmatory data to 
determine if an empirical relationship exists 
between the data or if discipline specific or 
contextual narrative (data) generally yields 
physical constants. 

For the purpose of this study the terms tag 
cloud, word cloud, and data cloud are used 
interchangeably. As noted, the theory of 
performance in the prototype design utilizes a 
correlation coefficient as an indicator of how 
closely student discussions are following stated 
learning objectives. A rising correlation 
coefficient would indicate a class discussion in 
close alignment with stated learning objectives; 
whereas, a sinking correlation coefficient would 
indicate a class discussion is off-track with 
stated learning objectives. In the latter case, 
instructor intervention would be required to alter 
the discussion in response to the stated learning 
objectives. 

Instructor interventions could take the form of 
restating goals and learning objectives for the 
class, guiding the discussion through leading 
questions, highlighting threads in the discussion 
that are in alignment with learning objectives, or 
radically altering the discussion via corrective 
narrative. The correlation coefficient serves as a 
dynamic indicator of instructional efficacy and 
can be visualized in a digital dashboard 
embedded in the LMS or as a stand-alone 
application. 

The experimental prototype developed for this 
study extracts a text string from any mediated 
discussion (e.g., wiki, blog, or discussion thread) 
as the input to a tag cloud engine. The output of 
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the tag cloud is analyzed and sorted based on 
the level at which the word appears in the cloud. 
Higher level words are considered emerging 
themes, dominant arguments, or narrative basis 
for the discussion. The standard or goal for the 
discussion is seen in the tag cloud output from 
the learning objectives. Code base evaluates the 
correlation coefficient of the student discussion 
to the learning objectives to determine how 
closely the discussion correlates to the goal. 

The outcomes of this study provide a basis for 
follow-on investigation into the use of Web 2.0 
technologies in guiding instruction and 
improving pedagogical efficacy. The act of 
relating relevant keywords to a site is known as 
tagging. Tagged data exists that could 
supplement the accuracy of diagnosing online 
discussion efficacy using tag cloud engines. The 
authors present case findings from two 
populations of online Information System 
students where discussion threads from one 
week on instruction addressing learning 
objectives related to the study of database 
technology were generated. The prototype 
model was applied to the week-long discussions 
from each population to determine which 
population most closely tracked with stated 
learning objectives. 

2. LITERATURE REVIEW 

The term Web 2.0 refers to services and user 
processes created with emerging Internet and 
Web open standards and technologies. 
Concatenation of maturing Web applications and 
technologies to create innovative, facilitative 
design for collaboration, knowledge creation, 
and information mediation (infomediation) 
represents a central Web 2.0 concept. 
Aggregation and brokering of user data, 
construction of social networks, creation of Web 
services, and exploration and discovery are 
driving goals in Web 2.0 initiatives. In effect, the 
old model of the Web as an information 
repository passively accessed by users changes 
to a platform for social constructs and 
collaboration, interaction and exchange, and 
personalized content ontologies (Torniai, 
Jovanovic, Gasevic, Bateman, & Hatala, 2008). 

According to Anderson (2007), the term Web 2.0 
is not best described by a set of technologies but 
as an idea encompassing individual contributions 
to content, knowledge construction using a 
"power of the crowd" methodology, large 
volumes of data and information, user 
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participation, open architecture, and network 
attributions. In 2004 Dale Dougherty, a vice- 
president at O'Reilly Media Inc., introduced the 
term Web 2.0 and defined it as using the Web as 
a platform to construct collaborative, user¬ 
centric content and interactive applications. 
Safran, Helic, and GutI (2007) posit that Web 
2.0 has coalesced with the eLearning domain. 
Following O'Reilly's introduction of the Web 2.0 
term, Stephen Downes introduced the term 
eLearning 2.0 (Wever, Mechant, Veevarte, & 
Hauttekeete, 2007). 

The primary attribute associated with Web 2.0 
technologies, and associated eLearning 2.0 
concept, involves a focus on making connections 
between learners and learning resources 
{connectivism) and the inclusion of social 
networking and Web 2.0 technologies as new 
elements of eLearning instructional design 
(Wever et al., 2007). As a result of 
implementing Web 2.0 technologies, learning 
spaces and communities of learners are created 
and social data can be utilized to best meet the 
instructional needs of a given learner population. 

Nascent Web technologies, now associated with 
Web 2.0, offer an opportunity to change 
development and delivery of instruction. Web 
2.0 is less a category of technologies and more 
an idea or design concept that supports 
constructivist approaches to eLearning. For 
example, the Community of Inquiry (Col) model 
(Garrison, Anderson, & Archer, 2000) optimizes 
synchronous and asynchronous computer 
mediated communication in a design focused on 
three core elements: social presence, cognitive 
presence, and teaching presence. eLearning 
environments utilizing Web 2.0 can play a key 
role in supporting discourse, based on the Col 
model. Parturient eLearning is accommodated 
not by a Web used only for component 
connectivity, but a Web used as a platform for 
development. 

Conceived by Jorn Barger in 1997, Weblogs 
(Blogs) refers to Web-based scrolls, presented in 
reverse chronological order, utilized as a 
mechanism for communication between 
interested user groups (Boulos & Wheller, 2007). 
Blogs are ideal for controlling a 1:M relationship 
between an instructor and a class of students 
(Ullrich, Borau, Luo, Tan, Shen, & Shen, 2008). 
Blogs contain posts and each post is generally 
tagged with one or more keywords. Associated 
tags allow the post to be cataloged based on a 
theme in a standard menu system. Meta-data 
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tags appear in close proximity to the posts and 
allow the user to navigate to other related posts 
(Alexander, 2006). Generally, blogging 
facilitates syndication, or the generation of feeds 
using RSS or, increasingly. Atom. Blog 
aggregators and special blog reading tools 
accept these feeds. The term blogosphere refers 
to the universe of bloggers who contribute to 
blogs in real time. 

Wikis were introduced by Bo Leuf and Ward 
Cunningham in 1995 as an online system to 
permit users to create, edit, revise, or link 
hypermedia. Ideal for collaborative work, the 
term wiki can be described as a knowledge 
management system used as collaborative 
media groupware. According to Ebner (2007), 
wikis have alternative functionalities to blogs. 
Wikis contain a history function, storing previous 
versions, and a rollback function, to restore 
previous versions. As a group work tool, wikis 
feature a simple, hypertext-style linking of 
pages to create navigation pathways. 

The term social bookmarking refers to a method 
for Web users to organize, store, manage, and 
search for bookmarks of resources online and 
has evolved into folksonomies which social 
bookmarking tools use as meta-data tags for 
search purposes. Essentially, folksonomies 
represent an ontology that has evolved from a 
community of practice where folksonomic meta¬ 
data is created by users who generate and 
attach related words to content. As a result, 
folksonomies interrelate learning content 
information. According to Boulos, Maramba, and 
Wheeler (2006), tools based on folksonomies 
are available to locate information related to 
specific research and capitalize on the 
observations and comments of other similar 
researchers. Folksonomies can identify a 
collection of resources that is evolving in concert 
with a specific research initiative. Folksonomical 
tagging illustrates a best-practice with respect to 
meta-data. Szomszor, Cantador, and Alani 
(2008) studied the correlation of user profiles 
using folksonomies and presented a framework 
to demonstrate cross-linking distributed user tag 
clouds to identify users separately on the Web. 

According to (Ullrich et al., 2008), social 
bookmarking services allow for the collection 
and annotation (i.e., tagging) of online content. 
This action enables a simple distribution and 
sharing of resources among a user community. 
Examples of social bookmarking sites include 
del.icio.us (http://del.icio.us). Furl 

(http://www.furl.net), Connotea 
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(http://www.connotea.org), and CiteULike 
(http:www.citeulike.org). Social bookmarking 
sites can be used as an online community tool to 
classify resources based on informally assigned, 
user-defined keywords or tags. Tags, when 
made public, can serve as a methodology for 
locating sites and other Web-based resources 
based on common or related keywords. Tags, in 
effect, serve as meta-data definitions for digital 
content and/or digital content objects. 

Moreover, sets or groups of tags (i.e., tagsets), 
can be visually displayed in a form of concept 
map known as a tag cloud. Tag clouds are useful 
in determining common or dominant themes 
from tagsets. Websites, documents, or other 
text-based content, such as the discussion in an 
online forum (e.g., blog, wiki, chat, or threaded 
discussion). According to Schrammel, Leitner, 
and Tscheligi (2009), tag clouds are used 
frequently to interact on the Web. As an adjunct 
outgrowth of tags, the concept and use of 
folksonomies (folk taxonomies) has increased. 
Based on the use of taxonomies to define and 
provide structural organization, folksonomies are 
developed by users as a collection of tags, 
created for personal use. Folksonomies involve 
the grouping of common user-created tags as a 
structured means of organizing and accessing 
digital content. Research tools using 
folksonomies as a methodology for locating 
related information are available. 

Moreover, tag clouds provide a helpful visual 
summary of content (Schrammel, Leitner, & 
Tscheligi, 2009). Szomszor, Cantador, and Alani 
(2008) studied the correlation of user profiles 
using folksonomies and presented architecture 
to demonstrate cross-linking distributed user tag 
clouds to identify users separately on the Web. 
Xexeo, Morgado, and Fiuza (2009) describe the 
output of a tag cloud with the term semantic 
field. The semantics illustrate and define the 
contextual meaning of the input text string. 
According to Flearst and Rosner (2008), tag 
cloud input primarily involves unstructured social 
data or annotations of information by authors 
where clouds are generated using query terms, 
word frequencies, category labels, or other 
heuristically determined algorithm. They also 
note that the primary value of the cloud is as a 
signal or marker of individual or social 
interaction with the contents of an information 
collection and functions as a suggestive device 
for some underlying phenomena. 
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Kuo, Hentrich, Good, and Wilkinson (2007) used 
tag clouds to summarize Web search results and 
found that tag clouds provide an overview of 
knowledge represented by an entire response 
and an interface to discover potentially relevant 
information hidden deep within the text string. 
In a study by Koutrika, Zadeh, and Garcia- 
Molina (2009), the researchers found that tag 
clouds can dynamically highlight the most 
significant concepts and hidden relationships 
within unstructured data. According to Bateman, 
Gutwin, and Nacenta (2008), clouds have been 
shown to assist in understanding data and 
semantic exploration. 

3. EXPERIMENTAL RESEARCH PROTOTYPE 
CASE 

Theory of Prototype Construction 
The prototype application was fashioned as a 
digital dashboard and consists of three primary 
modules: 

(i) data extraction and staging, 

(ii) a tag cloud engine, and 

(iii) correlation coefficient calculation and output 
rendering. 



Figure 1: High-level architecture for prototype 
design and construction 


As illustrated in Figure 1, a text string can be 
extracted from any social media communication 
forum, in this case the discussion threads from a 
LMS. The text string serves as the input to a tag 
cloud engine where the tag cloud output is 
organized by (minimally) the top three relative 
positions. Level one indicates the top level 
output, level two indicates the secondary level 
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output, level three indicates the tertiary level 
output, and so forth. 

Visually, the tag cloud output matches the 
organization of words into levels. In this 
research, the prototype evaluated words to 15 
levels. With more levels evaluated and scored, 
more data is available for calculation of the 
correlation coefficient, the final module of the 
prototype application. 

Use Case Applied to Prototype 
The student populations involved in this study 
included two sections of a graduate level 
Information Systems course on database system 
development. Both sections of the course were 
sampled in the fifth week of the term. During 
the fifth week of the term the control group 
(population 1) received no pedagogical 
facilitation or intervention, whereas the variable 
group (population 2) received daily pedagogical 
facilitation and intervention. The control group 
(N=15) and variable group (N=17) were 
composed of the following homogenous 
demographics: 

(i) 28-40 years of age, 

(ii) professional, adult students, 

(iii) technical undergraduate degrees, and 

(iv) at least four previous online courses 
completed. 

In this experimental research, three text strings 
acted as input to a tag cloud engine: 

(i) the learning objectives specified for the fifth 
week of instruction, 

(ii) the week five discussion forum for student 
population 1 taken from a LMS, and 

(iii) the week five discussion forum for student 
population 2 extracted from the same LMS. 

Text string output from a tag cloud engine for 
populations 1 and 2 were individually compared 
to the text string output from a tag cloud engine 
for the learning objectives. Populations 1 and 2 
were provided instruction online based on 
intended learning objectives. 

Two behavioral learning objectives were 
specified for the week of instruction utilized in 
this study and were stated as: 

After successful completion of this course, 
students will be able to: 
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1) Compare structured and agile 
development methodologies and, after 
comparing, evaluate for the most 
appropriate life-cycle methodology for a 
given database or information system 
project; and 

2) Utilizing a structured approach, apply a 
system development life-cycle 
methodology in construction of a 
database system. 


The tag cloud output from student population 1, 
the control group where no instructor facilitation 
or intervention occurred during the week, is 
featured in Figure 3. Based on tag cloud 
properties, dominant themes are noted by the 
term development at the top level, the term 
agile at the secondary level, and the terms 
database, methodology, and project at a tertiary 
level. Additional levels of word data for this 
population are shown in Table 1. 


Figure 2 features the tag cloud generated from 
the learning objectives captured as a text string. 
Based on tag cloud visual properties, dominant 
themes are defined by the terms system, 
development, and methodology at the top (most 
important) level, followed by the terms 
database, life, and cycle at a secondary level of 
emphasis. This output is noted in Table 1. 


cycle 


life 

methodology 

system 

Figure 2: Tag cloud engine results (learning 
objectives) 


^tabase . 

aeveropment 


agile applicofioi approach 

attribute? best bus i-ss changes 
complete customer 

v ie data database 


iterative 

methodology methods 

object phases pic is 
process programming 

project requirements 
software system 

team top Jov-1 under- and 

working 

Figure 3: Tag cloud engine results (Student 
population 1: control group) 



The tag cloud output from student population 2, 
the variable group where instructor facilitation 
and intervention occurred during the week, is 
featured in Figure 4. Based on tag cloud 
properties, dominant themes are noted by the 
term data at the top level, the terms 
development, database, and system at the 
secondary level, and the terms requirements 
and process at a tertiary level. Additional levels 
of word data for this population are shown in 
Table 1. 


agile approach 

businesr .;reate :-tomc 

data database 

ibms design 

development difronent 

impoi^i t iSSt 

manage methodology i iclhods 
model nt ^vork object 

performar ;e phase prOCGSS 

project requirements 

ser '“I software stage 
system user warehouse 
work 

Figure 4: Tag cloud engine results (Student 
population 2: variable group) 


5. RESULTS AND CONCLUSIONS 

Based on output from the tag cloud engine. 
Table 1 illustrates the word groupings by 
dominant theme for three text strings: 

(i) the stated learning objectives for the week, 

(ii) the week-long discussion forum for the 
control group, and 

(iii) the week-long discussion forum for the 
variable group. 
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Table 1 : Word groupings by dominant theme 


Learning 

Objectives 

Student 
Population 1: 
Control 
Group 

Student 
Population 2: 
Variable 
Group 

Level 

1 

System, 

Development, 

Methodology 

Development 

Data 

Level 

2 

Database, Life, 
Cycle 

Agile 

Development, 

Database, 

System 

Level 

3 


Database, 

Methodology, 

Project 

Requirements, 

Process 

Level 

4 


Process, 

Software, 

Design, 

System, 

Approach 

Agile, Design, 
Work 

Level 

5 



Methodology, 

Warehouse, 

Manage 


To achieve results and subsequent conclusions, 
the authors utilized a ranking scoring system 
based on the level at which each word data 
occurred where 100 represented the primary 
level, 90 represented the secondary level, etc. 
The integer 1 represents the lowest level where 
no occurrence exists. 

Table 2 illustrates the level and associated score 
for occurrence. The ranking scoring method 
appropriately awards higher level word 
positioning and assigns linearity to best 
accommodate use of p rank correlation 
coefficient to calculate the strength of linear 
relationships between the data. 


Table 2: Scores assigned by ievei of 
occurrence 

Level 

Score for occurrence 

1 

100 

2 

90 

3 

80 

4 

70 

5 

60 


Fifteen distinct terms, taken from the top five 
levels of word data in the three tag clouds were 
assigned scores based on the linear scoring 
method (Table 3). Two correlation coefficients 
were calculated: tag cloud output from the 
stated learning objectives and student 
population 1, and tag cloud output from the 
stated learning objectives and student 
population 2. 

The authors found that student population 1, the 
control group with no instructor facilitation or 


intervention, calculated significantly lower than 
student population 2, the variable group with 
instructor facilitation and intervention. Using r 
coefficient inclusive values of +1 (positive 
correlation) to -1 (negative correlation) as an 
indicator of pedagogical efficacy as measured 
through class topical discussions, lack of 
instructor facilitation and intervention is shown 
in the control group by a p score of .481. 
Evidence of instructor facilitation and 
intervention in the variable group is shown by a 
p score of .715. In practice, online class 
facilitators using a digital dashboard dynamic 
indicator of a weekly discussion's correlation to 
intended learning objectives could intervene 
appropriately to alter course discussions toward 
higher positive correlation. In large online 
classes, facilitators could save time reading long 
discussion threads by utilizing correlation 
coefficients as indicators to intervene and meet 
a pre-determined threshold of acceptable 
achievement in discussions. 


Table 3: Correiation coefficients 


Word 

Learnin 

9 

Objectiv 

es 

Student 
Populati 
on 1: 
Control 
Group 

Student 
Populati 
on 2: 
Variable 
Group 

1 

System 

100 

70 

90 

2 

Developm 

ent 

100 

100 

90 

3 

Methodolo 

gy 

100 

80 

60 

4 

Cycle 

90 

1 

1 

5 

Life 

90 

1 

1 

6 

Database 

90 

80 

90 

7 

Project 

80 

80 

1 

8 

Structured 

80 

1 

1 

9 

Agile 

80 

90 

70 

10 

Informatio 

n 

80 

1 

1 

11 

Apply 

80 

1 

1 

12 

Approach 

80 

70 

1 

13 

Appropriat 

e 

80 

1 

1 

14 

Comparin 

g 

80 

1 

1 

15 

Constructi 

on 

80 

1 

1 

Correlati 

on 

Coefficie 
nt to 
intended 
learning 
objective 
s 



0.48168 

0.71584 


Qualitative evaluation of the forum discussion in 
both populations revealed that students in 
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population 2, the variable group, engaged in 
discussion more directly related to achievement 
of the intended learning objectives. Moreover, 
their discussion was more detailed, cited more 
examples in support of arguments, and resulted 
in end-of-week summarizations reinforcing what 
had been learned. The qualitative evaluation 
served to reinforce the outcomes of this study. 

6. CONTRIBUTIONS AND FUTURE 
RESEARCH 

This experimental research contributes to the 
existing body of knowledge on application of 
Web 2.0 technologies in eLearning 
environments. Construction of a prototype based 
on the architecture represented in Figure 1 
demonstrates that social data, as collected in 
wikis, blogs, and LMS discussion forums, can be 
used to increase the efficacy of online and 
ground-based classroom instruction. Moreover, 
this research serves to inform educators of 
innovative uses and applications for Web 2.0 
technologies, specifically tag clouds. As a 
baseline study, this research serves as a 
foundation for additional exploration using an 
Information Systems approach to construction of 
mediated learning applications. 

Future research opportunities include collection 
of data in disciplines other than Information 
Systems to better understand the generalization 
of the application. Additionally, additional 
technology research into integration of the code 
and tag cloud engine with a LMS to create a 
digital dashboard as a component of the LMS. 
Currently constructed as a software 
development kit (SDK), the application 
programming interface (API) would benefit from 
further development. Other tools and utilities 
could be developed to provide analysis and 
reporting of the data in support of indicators 
such as the correlation coefficient. 

Finally, future research opportunities include 
prototype use in real-time. Students using 
laptops to blog during class on lecture related 
material could be sampled in short (60 second) 
intervals to determine at a group level the 
threshold of understanding based on learning 
outcomes. Online sample frequency also should 
be investigated further to determine the impact 
on pedagogical strategies and interventions. 
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